The technology described herein relates to the processing of computer graphics, and in particular to a method of and an apparatus for carrying out anti-aliasing when processing computer graphics.
As is known in the art, graphics processing is normally carried out by first dividing the output to be generated, such as a frame to be displayed, into a number of similar basic components (so-called “primitives”) to allow the graphics processing operations to be more easily carried out. These “primitives” are usually in the form of simple polygons, such as triangles.
The graphics primitives are usually generated by the applications program interface for the graphics processing system, using the graphics drawing instructions (requests) received from the application (e.g. game) that requires the graphics output.
Each primitive is at this stage usually defined by and represented as a set of vertices. Each vertex for a primitive has associated with it a set of data (such as position, colour, texture and other attributes data) representing the vertex. This data is then used, e.g., when rasterising and rendering the vertex (the primitive(s) to which the vertex relates) in order to generate the desired output of the graphics processing system.
Once primitives and their vertices have been generated and defined, they can be processed by the graphics processing system, in order, e.g., to display the frame.
This process basically involves determining which sampling points of an array of sampling points covering the output area to be processed are covered by a primitive, and then determining the appearance each sampling point should have (e.g. in terms of its colour, etc.) to represent the primitive at that sampling point. These processes are commonly referred to as rasterising and rendering, respectively.
The rasterising process determines the sample positions that should be used for a primitive (i.e. the (x, y) positions of the sample points to be used to represent the primitive in the output, e.g. scene to be displayed). This is typically done using the positions of the vertices of a primitive.
The rendering process then derives the data, such as red, green and blue (RGB) colour values and an “Alpha” (transparency) value, necessary to represent the primitive at the sample points (i.e. “shades” each sample point). This can involve, as is known in the art, applying textures, blending sample point data values, etc.
(In graphics literature, the term “rasterisation” is sometimes used to mean both primitive conversion to sample positions and rendering. However, herein “rasterisation” will be used to refer to converting primitive data to sampling point addresses only.)
These processes are typically carried out by testing sets of one, or of more than one, sampling point, and then generating for each set of sampling points found to include a sample point that is inside (covered by) the primitive in question (being tested), a discrete graphical entity usually referred to as a “fragment” on which the graphics processing operations (such as rendering) are carried out. Covered sampling points are thus, in effect, processed as fragments that will be used to render the primitive at the sampling points in question. The “fragments” are the graphical entities that pass through the rendering process (the rendering pipeline). Each fragment that is generated and processed may, e.g., represent a single sampling point or a set of plural sampling points, depending upon how the graphics processing system is configured.
(A “fragment” is therefore effectively (has associated with it) a set of primitive data as interpolated to a given output space sample point or points of a primitive. It may also include per-primitive and other state data that is required to shade the primitive at the sample point (fragment position) in question. Each graphics fragment may typically be the same size and location as a “pixel” of the output (e.g. output frame) (since as the pixels are the singularities in the final display, there may be a one-to-one mapping between the “fragments” the graphics processor operates on (renders) and the pixels of a display). However, it can be the case that there is not a one-to-one correspondence between a fragment and a display pixel, for example where particular forms of post-processing, such as downsampling, are carried out on the rendered image prior to displaying the final image.)
(It is also the case that as multiple fragments, e.g. from different overlapping primitives, at a given location may affect each other (e.g. due to transparency and/or blending), the final pixel output may depend upon plural or all fragments at that pixel location.)
(Correspondingly, there may be a one-to-one correspondence between the sampling points and the pixels of a display, but more typically there may not be a one-to-one correspondence between sampling points and display pixels, as downsampling may be carried out on the rendered sample values to generate the output pixel values for displaying the final image. Similarly, where multiple sampling point values, e.g. from different overlapping primitives, at a given location affect each other (e.g. due to transparency and/or blending), the final pixel output will also depend upon plural overlapping sample values at that pixel location.)
One problem that is encountered when processing graphics for display (when displaying computer generated images) is that the displayed image is quantised into the discrete pixel locations of the display, e.g. monitor or printer, being used. This limits the resolution of the image that is displayed and can produce unwanted visual artifacts, for example, where the resolution of the output display device is not high enough to display smooth lines. These effects are commonly referred to as “aliasing”.
FIG. 1 illustrates such aliasing effects. The lefthand side of FIG. 1 shows the image to be drawn, and the righthand side shows the actual image that is displayed. As can be seen, the desired smooth curve of the white object in fact has a jagged appearance on the display. This is aliasing. (In FIG. 1, each square represents a pixel of the display, and the crosses represent the points at each (x, y) pixel location for which the colour value for that pixel location is determined (sampled). For example, the pixel A in FIG. 1 is drawn as all white, because the colour sampling point for that pixel location falls within the white object. It should be noted that in FIG. 1 only sample crosses on the pixels of interest are shown, although in practice all pixels will be sampled.)
All aliasing artifacts that could be visible to the naked eye could be removed by using a sufficiently high resolution display. However, the resolution of electronic displays and printers is typically limited, and so many graphics processing systems use other techniques to try to remove or reduce the effects of aliasing. Such techniques are typically referred to as anti-aliasing techniques.
One known anti-aliasing technique is referred to as supersampling or oversampling.
In such an arrangement, there are plural sampling points (positions) per pixel of the final display, and a separate colour sample is taken for each (covered) individual sampling point (e.g. by rendering each sampling point as a separate fragment). The effect of this is that a different colour sample is taken for each sampling point of a display pixel that is covered by a primitive during the rendering process.
This means that plural colour samples are taken for each pixel of the display (one for each sampling point for the pixel, since a separate colour value is rendered for each sampling point). These plural colour samples are then combined into a single colour for the pixel when the pixel is displayed. This has the effect of smoothing or averaging the colour values from the original image at the pixel location in question.
FIG. 2 illustrates the supersampling process. In the example shown in FIG. 2, four sample points are determined for each pixel in the display and separate colour samples are taken for each sample point during the rendering process. (Each such sample can accordingly effectively be viewed as a “sub-pixel”, with each pixel in the display being made up of four such sub-pixels.) The four colour value samples (sub-pixels) for a given pixel are then combined (downfiltered) such that the final colour that is used for the pixel in the display is an appropriate average (blend) of the colours of the four colour samples taken for the pixel.
This has the effect of smoothing the image that is displayed, and, e.g., reduces the prominence of aliasing artifacts by surrounding them with intermediate shades of colour. This can be seen in FIG. 2, where the pixel A now has two “white” samples and two “black” samples and so is set to 50% “white” in the displayed image. In this way, the pixels around the edges of the white object are blurred to produce a smoother edge, based on, e.g., how many samples are found to fall on each side of the edge.
Supersampling in effect processes the screen image at a much higher resolution than will actually be used for the display, and then scales and filters (down samples) the processed image to the final resolution before it is displayed. This has the effect of providing an improved image with reduced aliasing artifacts, but requires greater processing power and/or time, since the graphics processing system must in effect process as many fragments as there are samples (such that, e.g., for 4× supersampling (i.e. where 4 samples are taken for each display pixel), the processing requirements will be four times greater than if there was no supersampling).
Other anti-aliasing techniques have therefore been proposed that, while still providing some improvement in image quality, have less processing requirements than full supersampling.
One common such technique is referred to as “multisampling”.
In the case of multisampling, multiple sampling points are again tested for each display pixel to determine whether a given primitive covers the sampling points or not when the image is rasterised into fragments (at the rasterisation stage). Thus the sampling point coverage of a primitive in a multisampling system is determined in a similar fashion to a “supersampling” system (and so the positions of the outer geometric edges of the primitives are still effectively “supersampled” (oversampled) in a multisampling system).
However, in the rendering process of a multisampling system, all the sampling points for a given display pixel that are covered by the primitive in question are allocated the same single, common set of colour value data (rather than each having their own separate set of data as would be the case for supersampling).
Thus, in multisampling, plural samples are again taken for each pixel that will make up the final display, but rather than determining a separate colour value for each sample when rendering the “pixel” (as would be the case for a full supersampling system), a single colour value is determined and applied to all the samples for a display pixel that are found to belong to the same object in the final image. In other words multisampling calculates a single colour value for a given display pixel for a given object in the scene, which colour value is applied to (reused for) all samples (subpixels) of the display pixel that are covered by that object (in contrast to supersampling where a separate colour value is determined for each sample).
Because only a single colour value is used for multiple samples for a given display pixel, multisampling is less processing intensive than supersampling and therefore can allow faster processing and performance than supersampling. However, there is a reduction in the quality of the displayed image as compared to supersampling, since although objects' edges are still sampled at a higher resolution, colours are not.
Notwithstanding this, many graphics processing systems use a multisampling anti-aliasing technique, as multisampling can in general provide adequate (and improved as compared to when there is no multisampling or supersampling at all) anti-aliasing in a rendered image, but without the significant extra processing and computational burden that full supersampling entails.
Many graphics processing systems are configured to be able to support a given level of multisampling anti-aliasing (MSAA) “natively” (i.e. are optimised for performing a given level of MSAA). For example, a tile-based graphics processor can be configured to support 4×MSAA at full speed by configuring the rasteriser to be able to test four sample points per (output) pixel per clock cycle, the blenders to be able to blend four samples per clock cycle, the depth (stencil) test units to be able to do depth/stencil checking for four samples per clock cycle, sizing the tile buffers for four samples per output pixel, configuring the tile writeback to be able to average four samples per pixel written back to main memory, and configuring any calculation units to be able to select the correct sample point for centroid mapping, etc., on the basis that 4×MSAA will be being used. Such a configuration can then allow the graphics processor to support 4×MSAA with little or no performance penalty.
However, it is now becoming more common for graphics APIs to require higher levels of anti-aliasing. For example, 8×MSAA is a required feature for Direct3D 11 compliance. One way to address this would be to configure a graphics processor to “natively” support the higher level of anti-aliasing, e.g. to support doing eight rather than four of everything per cycle. However, this would be relatively expensive, e.g. in hardware terms.
It would also be possible to approximate 8×MSAA by using 4×MSAA plus 2× supersampling (2×SSAA). However, this would only be an approximation to “true” 8×MSAA. For example, if using centroid mapping (where varyings are evaluated at the approximate centre of the covered sampling points associated with a fragment, instead of at the centre of the (output) pixel (or the centre of all the sampling points, whether covered or uncovered), then using 4×MSAA plus 2×SSAA would evaluate varyings at different positions to “true” 8×MSAA, potentially producing totally different results. Since Direct3D 11 requires an exact and predefined set of sample points for 8×MSAA, such as approximation would not be compliant with the Direct3D 11 standard.
The Applicants believe therefore that there remains scope for improved techniques for supporting higher levels of anti-aliasing in graphics processing systems.
Like reference numerals are used for like components where appropriate in the drawings.